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The scope of this bibliography has been limited to 
literature dealing with the design, testing and evaluation of 
information storage and retrieval systems. A number of papers 
describing the implementation of specific systems were selectively 
Included because they contained substantial or innovative material on 
evaluation; hovever, the bibliography generally emphasizes techniques 
applicable to a wide variety of systems. Similarly, articles dealing 
with specific products and services have been included when they 
show3d broad implications for evaluation. On the other hand, 
discussions of specific library systems have been excluded as they 
were considered outside the scope of the bibliography. You will not, 
therefore, find such articles as those dealing with "objective tests 
of library performance." The last comprehensive bibliography in the 
field was compiled by Madeline Henderson and published in 1967. For 
that reason, the earliest citations in this compilation are for 1967. 
(Author/3J) 
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INTRODUCTION 



Design 

The scope of this bibliography has been limited to literature dealing with 
the design, testing and evaluation of information storage and retrieval systems. 
A number of papers describing the implementation of specific systems were selec- 
tively included because they contained substantial or innovative material on 
evaluation; however, the bibliography generally emphasizes techniques applicable 
to a wide variety of systems. Similarly, articles dealing with specific products 
and services have been included when they showed broad implications for evaluation. 
On the other hand^ discussions of specific library systerr-s have been excluded as 
they were considered outside the scope of the bibliography. You will not, 
therefore, find such articles as those dealing with "objective tests of library 
performance.. " 

The last comprehensive bibliography in the field was compiled by Madeline 
Henderson^ and published in 1967. For that reason, the earliest citations in 
6ur compilation ore for 1967, although no sys':ematic atterr.pt was made to remove 
those citations duplicating Henderson's v;ork. The year 1972 was considered an 
expedient limiting date, thereby permitting the bibliography to be current and 
yet remain a reasonable size. Final preparation of the bibliography began in 
early January 1973, thus excluding publications from late 1972. The authors 
felt that this cutoff was necessary if the bibliography was to be printed in 
1973. 

Methodology 

The basic tools used in compiling this material were: Research in 
Education (via ERIC/CLIS on-demand searches and monthly accession lists). 
Library Literature , and Information Abstracts . Liberal use was made of bibli- 
ographies in the Annual Revi3w of Infoxnation Science and Technology (Carlos A. 
Cuadra, ed.) All material (monographs, technical reports^ journal articles, 
rejoinders, etc.) have been included regardless of possible duplication of 
content in the hope that this degree of noisiness in our approach will increase 
the potential user's ability of actually obtaining a copy of the material de- 
sired. Occasionally both PB and ED numbers have been available; when this has 
occurcd both have been included. Also, if an author has flatly stated that two 
publications are id'^ntical, this has been noted. 



"Madeline M. Henderson, Evaluation of Information Systems - a Selected 
Bibliography with Infonr . ative Abstracts . National Bureau of Standards, 
Washington, D.C. , December, 1967. (NBS-TN-297, ED 061497). 
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Classification Scheme 

Each citation is listed unuer one (and only one) category within the 
classification schedule. (See page 5 and 6 for the categories used in the 
classification scheme.) Although this limits the access to a citation to 
only one point, the authors felt that a multiple access point bibliography 
with such a limited scope would become highly repetitious. The hardest 
material to classify, of course, was the published monographs. These, 
therefore, have been crouped together outside the claosification scheme. 

Some General Imoressions 

, 

We sta|:ted this bibliogra'^\y in the hope that we would gain a deeper 
understanding of the problems of evaluating information- systems . At the 
begitining, we had expected to road, and in the words of Licklider, "interact 
deeply," with this literature. ' It proved far too large, and our industry and 
the tifne available did not permit us that luxury. We are a little disappointed 
at being unable to carry through our original plan. 

We are also somewhat disappointed in the literature that we have been 
inspecting over these months. It is piecen.eal, nonc\amulative , and in a rather 
fundamental sense, moribund. It is a reflection upon our ov/n field that there 
is not more integration of work, greater breadth of perspective in this area, 
and better scholarly analysis and review of the work. The stagnant quality of 
the literature has another origin, namely, the apparent lack of any recent and 
substantial funding to do the massive studies required to make basic further 
contributions to this field. Therefore, there is a growing literature that 
consists largely of reanalyses of, for example, the Cranfield data of the ap- 
plication of indexing to retrieval,. 

fe 

The literature which this bibliography contains seems to have advanced 
little beyond the work of Cleverdon, Guiliano, Salton and Swets in the area 
of retrieval; little beyond the work of Bradford, Brookes, Fairtliome and 
Price in the structure of information; and little beyond the work of Orr, Allen, 
Kent, Wolek, Garvey and Griffith, in the analysis of the benefits of informa- 
tion to the user. 

The most distinguished effort is the work of the group of researchers 
and information specialists that clustered in Westat between 1968 and 1970, 
meaning, of course, Lancaster, King, and Bryant, and in the new focus of 
their work on cost-effectiveness. The single most influential article, the 
1971 JASIS article by Lancaster, 2 ^^as hailed by the community with a si jh of 
relief. The information science community has long received beatings on the 
sinple straight-forward basis that they could not demonstrate much substantial 



P.W, Lancaster "The cost-effectiveness analysis of information 
retrieval and dissemination systems," Journal of the American 
O „ Society for Information Science 22 (January-February 1971) 12-27. 
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benefits to users ^ and Lancaster offered them an alternative which can be 
summarized, perhaps a little negatively, as "we may not be doing anything of 
Ulitmate value (and it may not be up to us to discuss 'ultimates') but what 
we are doinc; we are doing very .^ell." 

The results* are not in on whether this new approat^ will substantially 
affect the relation of the inforniation science corranunity to its funders. 
(We are not optimistic.) it has clearly created great benefits within the 
conunimi cy in terms of calling attention to the factors common to mos* doc\iment 
systems and by emphasizing the intrinsically probabilistic nature of information 
and its handling. The only effect of the cost-effectiveness approach that 
We deplore is that it effectively separates an "evaluation" literature from 
a "user" literature, to the possible det . ment of both. The concept of the 
cost-effectiveness of information systems appeared just at the point that it 
was becoming very evident 1) that most information systems created for 
scientists took little account of either the informal or conceptual struct ure 
of science , which clearly guides the scientist's search for information, ^ 
2) that only the most preliminary data on the information habits and neea^ of 
workers in applied science and technology had been gathered. In general, the 
"user" research had begun to raise q'^estions about the "document" and 
"retrieval" orientations of precisely the type of systems the V/estat group 
were best prepared to evaluate. 

Cleverdon wrote in a recent review ( Journal of Documentation of the 
major monograph of the VJestat school of evaluation . 3 "it is clear that the 
authors thought deeply about the subject without doing very much practical 
work...." However, this group has done a great deal of practical work, or 
what passes for it to government funders, on evaluations of real systems. 
More in line with our ov;n thinking, Swanson wrote (in Library Quarterly ) 
regarding the same monograph, "in a sense it is a handbook ,. almost a 'cookbook', 
of what to measure and how to utilize such measurements (to describe) the 
performance of the system. Some attention is also given to cost analysis, 
with detailed examples." The implicit contrast between these comments arises 
naturally from thj Westat approach. On one hand, the approach does not 
really help the practitioner in his rough and ready dialogue, filled with all 
sorts of unspoken assumptions and biases, between himself and his governing 
body or agency. This interaction, which the autho" (Griffith) has tor 
pedagogical reasons called the "evaluation dialogue" almost never rises above 
the restrictions of "local ground rules," even for the largest information 
centers an^ services. 



to. W. King and E. C. Bryant, The Evaluation of Infcrm^.tion 
Services and Products . Washington, Information Resources Press, 
1971. 



The other, and much more sv/eeping criticism, is best understood at a 
very general level: Information systems rarely offer a single service or 
product and there are, presently, no formal ways of equating cind optimizing 
total output. Similarly, the user is rarely satisfied with a single product • 
or service, and there are, presentl y^, no fo rmal ways of equating and optimiz- 
ing total desired input to the user. With this as a given, the information 
practitioner insures, basically, that the few facts he gathers and the judge- 
ments he makes are the facts and judgements welcomed by the governing grc^p. 
Again, if you accept the assumptions regardin^^ information output (for the 
system) and information input (to the user) , the use of elaborate techniques, 
however detailed, will nearly always be -inconclusive. 

Perhaps the real force behind the development of this literature is -that 
we have no more than the old practitioner-governing body dialogue, only th:..s 
time the practitioner heads a large system and the governing body is the 
Federal government. With that level of resources, the facts required and 
judgements rendered may be very complex indeed, even though the outcome may 
still be inconclusive. 

Finally the reader, if he wishes to get an understanding of the state- 
of-the-art and professional climate of the present, should read the works cf 
King, Bryant, and Lancaster with care. They are certainly the literature to 
which future work will react. 
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Classification Scheme 

I. Systems analyses of information systems 

A. General evaluation techniques 

B. General design teclmiques 

C. Modelling and simulation techniques 

D. System performance functions 

Cost-effectiveness techniques 

(The economics of information retrieval) 

II. Information system components 

A. Evaluation measures 

1) Relevance and recall 

2) Other retrieval effectiveness measures 

* 3) Applications of statistics to other aspects of systems 

B. File structure and design 

C. Secirch strategies 

D. Indexing languages and systems 

1) Conceptualization .and methodology 

2) Performance and effectiveness 

3) Evaluation 

4) Language devices 

5) Indexer consistency 
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III. Information products and services 

A. User/system interface 

B. Current awareness and selective dissemination of 
information (SDI) systems 

C. Real-time r interactive system*' 

D. Evaluation of other specific information products and/or 
services 
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PUBLISHED BOOKS 

The following published books are either wholly devoted or largely related 
to the evaluating of infoxnaation systems: 

Kent, Allen. Information Analysis and Retrieval . Becker and Hayes, New 
York, 1971, 367 pp. 

Kenti Allen; Taulbee, Orrin E,; Belzer, Jack; Goldstein, Gordon D, , eds. 
Electronic 'Handling of Information: Testing and Evaluation . John I, Thompson 
& Co., Washington, D. C., 1967, 313 pp. 

King, Donald W.; Bryant, Edward C. The Evaluation of Information Serv- 
ices and Products . Information Resources Press, Washington, D, C., 1971, 
306 pp. 

Kochen, Manfred, ed. The Growth of Knowledge; Readings on Organization 
and Retrieval of Information . John Wiley and Sons, New York, 1967, 334 pp. 

Lancaster, Frederick Wilfrid. Information Retrieval Systems; Character - 
is tics, Testing and Evaluation . John Wiley and Sons, New York, 1968, 222 pp. 

Matthews, Don Q. The Design of M an f^ement Information Sys terns ♦ Auerbach, 
Princeton, New Jersey, 1971, 221 pp. 

Morse, Philip M. Library Effectiveness; A Systems Approach . MIT Press, 
Cambridge, Massachusetts, 1968, 207 pp. 

Kubinoff, M. Information Systems Design . Moore School of Electrical 
Engineering, Philadelphia, Pennsylvania, 1971, 260 pp. 

Salton, Gerald. Automatic Information Organization and Retrieval . 
McGraw-Hill, New York, 1968, 51A pp. 

System Development Corporation. System Study of Abstracting and 
Indexing . Management Information Services, Detroit, 1970, 228 pp. 

Szweda, Ralph A. Information Processing Management . Auerbach, Princeton, 
New Jersey, 1972, 632 pp. 

Walker, Donald E., ed. Information System Science and Technology . John 
I. Thompson & Co., Washington, D. C. 



JOURNAL ARTICLES AND TECHNICAL REPORTS 

I. SYSTEMS ANALYSES OF INFORMATION SYSTEMS 

A. General Evaluation Techniques. 

Andrus, R. R. "Approaches to information evaluation," MSU Business 
Topics . (Summer 1971) 40-46, 



8 



ERIC 



Back, Harry B. "What information dlssenination studies Imply concern- 
Ing the design of on-line retrieval systems," Journal of the American Society 
for Information Science 2:3 (May-June 1972) 156-163. 

Cleverdon, Cyril W. The Critical Apprai s al of Information Retrieval 
Systems . The College of Aeronautics, Cranfield, Bedford, England, September 
1968. (Paper prepared for the International Congress of FID^ Moscow). 

Cleverdon, Cyril W. "Design and evaluation of information systens'* 
In: Annual Review of Information Science and Tech n ology . Volume 6. Edited 
by Carlos A. Cuadra. American Society for Information Science, Washington, 
D. C, 1971, A3-73. 

Cleverdon, Cyril W. "Progress in documentation: evaluation of infor- 
mation retrieval systems." Jo urnal of Docume nta tion 26:1 (March 1970) 55-67. 

■ \ 

. Comparative Systems Laboratory. An Inquiry into TGs_ting_ of Informa- 
tion Retrieval Systems . 3 vols. Edited by Tefko Saracevic and others. Case 
Western Reserve University, Cleveland, 1968, (PB 179290 or ED023421, IB 180952 
or ED 0270A2, PB 180952 or ED 0270A1). 

Davis, Ruth M. \ "Exparir.entat ion and evaluation: the ch:^l].enge . In: 
Electronic Handling of Information: Testing and Ev aluation. Edited by Allen 
Kent, Orrin E. Taulbee, Jack Belzer, Gordon D. Goldstein. John I, Thompson & 
Co., Washington, D. C, 1967, 3^12. 

Edwards, Ward; Phillips, Lawrence D.; Hays, William L. ; Goodman, 
Barbara C. , "Probalistic information processing systems; design and evalua- 
tion." IEEE Transactions on Systems, Science and Cybernetics ." A:3 (1968) 
246-265. 

Good, I. J. *'The decision theory approach to the evaluation of 
information retrieval systems^ Information Storage and Retrieval , 3:2 (April 
1967) 31-34. 

Henderson, Madeline M. Evaluation of Information Systems - A Selected 
Bibliography with Informative Abstracts . National Bureau of Standards, Wash- 
ington,^. C, December 1967. (NBS-TN-297, ED 016497, LI 000084). 

Herner, Saul. "System design , evaluation and costing." Special 
Libraries 5*8 (October 1967) 576-581. 

Hillraan, D. J. Prototype Retrieval S yb<-em Develo p ment Within an In - 
fortnation Resource Laboratory . Lehigh University, iiethlehcm, Pennsylvania, 
January 1970, 38 p. 

Hooper, R. S. Dev?*? <^pment end Application of a T ech nique for the 
Evaluation of Document Retrieval Systems . American University, Washington D. C, 
25 May 1967. 

Katter, Robert V. "Design and evaluation of informatior. systems." In: ^ 
/nnual Review of In fo rmation Science and Tech nology. Volume 4. Edited by 
Oarlos A. Cuadra. Encyclopaeuia Britannica, Chicago, 1969, 31-70. 
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Keith, Nathan R. .**A general evaluation rriodel for an information stor- 
age and retrieval system.** Journal of the American Society for Tnf ormati^^ri 
Science 21:4 (July-August ^ 1970) 237-239. 

King, Donald W. ^'DevSign and evaluation of information systems." In: 
Annual Review of Information Science and Technology . Volume 3. Edited by 

Carlos A. Cuadra. Encyclopaedia Britajinica, Chicago, 1968 • 61-103. 

■ ; • . \ 

Kinkade, Robert G. Desirable Characteristics of a Scientific Publica- 
tlon > American Phychcjlogical Association, Washington, D. C, April 1970, 19p. 
(OCMD-4-70-TR-3, ED038995, LI001922). 

Kriebel, Charles H. The Evaluation of Management Information Systems . 

National Technical Information Service, Washington, D. C. September 1970, 26p. 
^AD 723 083) . 

Kriebel, Charles H. I:: formation processing and programmed decision 

systems.'' Management Science;^ TheoLy 16 (November 1969) 149-164. - 

Lancaster, Frederick Wilfrid "Aftermath of an evaluation." Journal 
of Documentation , 27:1 (March 1971) 1-10. 

Lancaster, Frederick Wilfrid "Design and evaluation of information 
"systems," ^ In; Annual Review of Information Science and Technology , Volume 
5. Edited by Carlos A. Cuadra, Encyclopaedia Britannica,_ Chicago, 1970, 33-70. 



^ Lancaster, Frederick Wilfrid. "Evaluating the performance of a "Mrge 
computerized information system." Journal of the American Medical Association 
207 (6 January 1969) 114-120. 

Lancaster^ Frederick Wilfrid. "Evaluating 'the performance of a large 
operating retrieval system. In: Electronic Handling of Information; Testing 
and Evaluation . Edited by Allen Kent, Orrin E. Taulbee, Jack Belzer, Gordon D. 
Goldstein. John I* Thompson & Co., Washington, D.C., 1967, 196-216. 

Lancaster, Frederick Wilfrid. "Evaluation of information retrieval 
systems." In: Encyclopaedia of Library and Information Science . Volume 8. 
Edited by Allen Kent and Harold Lancourt. Marcel Deljjcer, New York, 1972, 
234-259. 

Lancaster, Frederick Wilfrid; Gillespie, C. J. "Design and evaluation 
of information systems," In: Annual Review of Information Science and 'Tec h- 
nology , Voluae 5. Edited by Carlos A. Cuadra. Encyclopaedia Britannica, 
Chicago,' 1970, 33-70. : ^ 

McCarn, Davis B.; Stein, Charles R. "Intelligence systems evaluation." 
In': Electronic Handling of Informations Testing and Evaluation , Edited by 
Allen Kent, Orrin E. " Taulbee, Jack Belzer, Gordon D; Goldstein, John I. 
Thompson & Co., Wa^^ington, D.C. , ^^1967 , 109-122. 

Martyn, John. ''Evaluation of information handling systems." Agl ib 
Proceedings. -21. (August 1969) 317-324. 

Martyn, John. "Evaluation of . specialized Information centers." 
'^ Information Scientist 4 (1970) 123-135. 
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Meetham, A. "Communication theory and the evaluation* of infor- 

mation retrieval systems." Information Storage and Retrieval 5 (October i969) 
129-134. 

Papier, L. S. "Criteria and protocol fqr intermediate evaluation of in- 
fonnatlDn systems by their operative iianager." In: AMERICAN SOCIETY FOR IN- 
FORMATION SCIENCE. Annual Meeting. 34th, Denver, Colorado, 7-11 November 1971. 
Proceedings. Volume 8: Communication for Decision-Makers. Edited by Jeinhe B. 
North. Greenwood Publishing Company, Westport, Connecticut, 1971, 83-90. 

Plrog, W. "Efficiency of -information activities, evaluation criteria, 
method and indices." In: All Union Institute for Scientific and Technical In- 
formation, On Theoretical Problems of Informatic s. Moscow, 1969, 93-105. 

Pratt, A. D. "Objectives and performance evaluation of information 
systfems." In: AMERICAN SOCIETY rOR INFORMATION SCIENCE. Annual Meeting. 31st, 
Columbus, Ohio, 20-24 October 1968. Proceedings. Volume 5: Information Trans- 
fer. Greenwood Publishing Corp-, New York, New York, 1968, .323-325. 

Rees, A. M. "Evaluation of information systems and services." In: 
Annual Review of Information Science and Technology . Volume 2. Edited by 
Carlos A. Cuadra. Interscience, Inc., New York, 1967, 63-68. 

Richmond, P. A. "The final report of the Comparative Systems Labora- 
tory: a review." Journal of the American Society for Information Science 21:2 
(March 1970) 160-162. 

Romerio, G. F.; Cavara, L. "Assessment studies of documentation 
systems." Information Storage and Retrieval 4:3 (August 1968) 309-325. 

* Ross, Larry Lynn. The Effectiveness of Two Systems for Delivering Oc - 
cupational Information: A Comparative Analysis . Oregon University, Eugene, 
Oregon, 1971, 121 pp. (ED 055332; Master's Thesis). 

Salton, Gerald, "Generalized effect ond the retrieval evaluation for 
large collections." Journal of the American Society for Information Science 23 
(January 1972) 11-22. . 

Sal ton, Gerald, Information Storage and Retrieval, Reports on. L valuation 
Procedures and Results^ 1965-1967 . Department of Computer Science, Cornell 
University, Ithaca ,^New York, January 1968. (ISR-13)* 

Sal ton, Gerald. -Reports on Analysis, Search arid Iterative Retrieval . De- 
partment of Computer Sciences, Cornell University-, Ithaca, New York, October 1968. 
(ISR-14). 

Saracevic, Tefko. "Quo vadis test and evaluation.'' In: AMERICAN D0CU?1EN 
TATION INSTITUTE. Annual Meeting. 30th, New York, New York, 22-27 October ' 1967. 
Proceedings. Volume 4: LeVels of Interaction between Man and Information. Edit- 
ed by Paul J. Fasana. Thompson Book Company, Washington, D.C.; Academic Press,. 
London, England, 1967, 100-104. ' 
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Saracevic, Tefko. "Selected results from an inquiry into testing of 
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InfoTOacian systems,*' Journ.n l of the American Society for Information Scieace 
•22:2 (March 1971) 126-139. Ulso, In: Introduction to Information Science , 
Edlced by Tefko Saracevic. Bowker, New York, 1970, 665-681.) 

Saracevic, Tefko; Rees, Alan M. "Towards the control of variables 
in inforciatloa retrieval expcrimeL tation/' Journal of Documentation 23 (March 
1967) 17-19* ~ ~ 

Sbumway, R, H. "Some estimation problems associated with evaluating 
lid format Ion retrieval systems." In: Evaluation of Docinient Retrieval Systems : 
>kij^!lj^JL^_._PgJ^^P M easurement, Technical Paper s, Westat Research, Inc., 

Bethesda, llaryland^ 31 December 1968, Chapter 3.1. 

Smithy A, > Jr. Studies in Information Systems; Effectiveness of 
Infor tDation Systems. Lehigh University, Bethlehem, Pennsylvania, June 1972, , 
66 pp. 027). 

StattCel» L. "On a quantitative approach to improving docuraent re- 
trieval perfortDance, " Information Storage and Retrieva l A (August 1968) 281- 
286* 

Swets^ John A. "Information retrieval systems." In: The Grox^th of 
Kr>ouled eg: Rea dings on Organisation and Retrieval of Information . Edited by 
Maatred Kochen. John Wiley & Sons, 196^^, 174-184. 

Taulbee, Orrin E. "An approach to comprehensive evaluation." In: . Elec- 
tronic Handllnp o f Information: Testing and Evaluation . Edited by Allen Kent, 
Orrin Taulbee, Jack Beizer, Gordon D. Goldstein. John I. Thompson & Co., 
Washington, D.C.> 1967, 217-230. . 

Treu» Sigfried, "Testing and evaluation: review." In: Electronic 
Bajpdiing of Information: Testing and Evalua.tion . Edited by Allen Kent, Orrin E. 
Xaulboe^ Jack Belzer, Gordon D. Goldstein. John I. Thompson & Co., Washington, 
1967. 

Van LoOjK M. "Decision criteria and evaluation models for information 
retrieval systems." In: Integrative Mechanisims in Literature Growth . Final 
B^port* Mental- Health Research Institute, University of Michigan, 1970.^ 

i 

VlckeVy, B. C* "In the last analysis all evaluation is subjective." 
Hethodology in Research. Aslib Proceedings 22 (1970) 597-606. 

' . - I 

Welch, John T. , Jr. "Measures and criteria for evaluation.^' In: 
Electronic Handling^ of Information: Testing and Evaluation . Edited by Allen 
Kent* Orrin E. Tauibee, Jack Belzer, Gordon D. Goldstein, John I,. Thompson & 
Co*, Washington, D..C., 1967 231^236, . 

Westat Research, Inc. Evaluation of Document Retrieval Systems: Liter- 
ature Perspective, Measurement, Technical Papers .. Bethesda, Maryland, 1968, . 
159 pp. (PB 182 710). 

Westat Research, Inc. Procedural Guide for the Evaluation of Document 
Retrieval System s. Bethesda, Maryland, 1968, 216 pp. (PB 182 711). 



12 



ERIC 



Whittenburg, J. A.; Schumacher, A. W. An Information System Plann ing 
Guide: Preliminary Development and Checkout > Whittenburg, Vaughn Associates, 
Inc., Alexandria, Virginia, February 1968. (PB 177 601). 

Wilcox, Richard H. "Conclusions on evaluation and evaluation of con- 
clusions." In: Electronic Handling of Information; Testing and Evaluation. 
Edited by Allen Kent, Orrin E. Taulbee, Jack Belzer, Gordon D. Goldstein. 
John I. Thompson 9 Co., Washington, D.C. 1967, 301-308. 

•Wilkin, A. P.; Reynolds, R.; Robertson, S. E. "Standard times for infor- 
mation systems a method for dat^i collection and analysis." Journal of Documenta - 
tion 28:2 (June 1972) 131-150. 

Wooster, Harold. "An information analysis center effectiveness chres- 
toraathy." Journal of the American Society for Information Science 21 (March- 
April 1970) 149-159. 

B. General Design Techniques, 

Borko, Harold. "Design uf informatibn systems and services." In: 
Annual Review of Information Science and Technology . Volujie 2. Edited by 
Carlos^'A. Caudra. Interscience, Inc., New York, 1967, 35-61. 

Center for Information Services, Phase II: Detailed System Design and 
Planning . 7 Parts. Institute of Library Research, Los Angeles, California, 1971, 
(ED 057 P06 through ED 057 812). 

Chamis, Alice Yanosko. "The design of information systems: the use of 
systems analysis." Special Libraries 60 (January 1969) 21-31 • 

Cooper, W. S. "On deriving design equations for information retrieval 
systems." Journal of the American Society for Information Science 21:6 (1970) 
385-395*. 

Davis, Ruth M. "The national biomedica? communications network as a 
developing structure." In: Proceedings of the Conference on Interlibrary Com - 
municrations and Information Networks . Edited by Joseph Becker. American Li- 
brarj^Association, Chicago, Illinois, 1970. (ED 057875). 

Salton, Gerald; Keen, E. M- ; Lesk, M. "Design of experiments in auto- 
inatic information retrieval." In.: The Growth of Knowledge: Readings on Organi - 
zation and Retrieval of Information . Edited by Manfred Kochen. John Wiley 6c 
Sons, 1967, 336-351. 

C. Modelling and Simulation Techniques. 

Baker, N.R.; Nance, R.E. "The use of simulation in studying informa- 
tion storage and retrieval systems." American Documentation 19:4 (October 1968) 
363-370. (Presented at the 32nd National Meeting of the Operations Research 
Society of America, Chicago, 111., November 1967). 

Biegel, J. E.; Sargent, R. G.; Foster, G. Large Scale Information 
Processing Systems . Volume II: Systems, model building, simulation and evalu- 
ation. Final Report. Syracuse University^ Syracuse, N. Y., 1968 (RADC-TR-67- 
A98-V01 2; AD 674053). 
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Bhat, U. Narayan: and others* Information Networks; a Probablistlc 
Model for Hierarchical Messa g e Transfer , Computer Science/Operations Research 
Center, Southern Methodist L.;iversity, Dallas, Texas, 1971 (TR-CP-710023; ED 
060 863). 
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